Achieving Transaction Scaleup on Unix

Constructing scalable high-performance applications on commodity hardware running the Unix operating system is a problem that must be addressed in several application domains. We relate our experience in achieving transaction scaleup on Unix for a high-performance OLTP system intended for Service Control Points (SCPs) in a telephone switching network. We view the requirements of SCPs as prototypical requirements of a class of applications that cannot be properly handled by today's commercial DBMSs. In addition to high throughput and low response time, SCPs require transaction scaleup on standard hardware and software architectures. Using a main-memory DBMS to obtain high throughput, we focus on the problem of achieving transaction scaleup on a cluster of workstations running Unix while constrained by the low response time requirement of the SCP application. The use of a main-memory DBMS causes the throughput and response time to be much more sensitive to the cost of messages and more susceptible to the formation of convoys than they would have been with a disk-based DBMS. We relate the various experiments and discoveries of what goes on underneath the covers in Unix that led to our choice of architecture, inter-process communication mechanisms and mode of running the system to achieve transaction scaleup under the constraint of low response time. This experience should provide valuable information to anyone trying to build a scalable high-performance application on Unix.